Medical information processing apparatus, x-ray diagnostic apparatus, and medical information processing program

ABSTRACT

A medical information processing apparatus of an embodiment includes an image acquiring unit, an event acquiring unit, a managing unit, and an output unit. The image acquiring unit sequentially acquires medical images during a treatment procedure for a subject. The event acquiring unit sequentially acquires events in the treatment procedure on the basis of the medical images during the treatment procedure. The managing unit manages the medical images and the events, in association with temporal information on the treatment procedure. The output unit outputs the medical images such that relations between the medical images and the events are able to be known.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2020-128499, filed on Jul. 29, 2020; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments relate to medical information processing apparatuses, X-raydiagnostic apparatuses, and medical information processing programs.

BACKGROUND

In treatment procedures for subjects, medical images acquired throughthe treatment procedures are displayed to support users, such as medicaldoctors who perform the medical procedures. Such medical images areretrieved by the users or other staff, from multiple medical imagesacquired through the treatment procedures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a configuration ofa medical information processing system according to a first embodiment;

FIG. 2 is a block diagram illustrating an example of a configuration ofan X-ray diagnostic apparatus according to the first embodiment;

FIG. 3A is a diagram illustrating an example of an event acquiringprocess according to the first embodiment;

FIG. 3B is a diagram illustrating an example of the event acquiringprocess according to the first embodiment;

FIG. 3C is a diagram illustrating an example of the event acquiringprocess according to the first embodiment;

FIG. 4 is a diagram illustrating an example of the event acquiringprocess according to the first embodiment;

FIG. 5 is a diagram illustrating an example of display according to thefirst embodiment;

FIG. 6 is a diagram illustrating an example of a related image acquiringprocess according to the first embodiment;

FIG. 7 is a diagram illustrating an example of display according to thefirst embodiment;

FIG. 8 is a diagram for explanation of hierarchical display of eventsaccording to the first embodiment;

FIG. 9A is a diagram illustrating an example of display according to thefirst embodiment;

FIG. 9B is a diagram illustrating an example of display according to thefirst embodiment;

FIG. 10A is a diagram illustrating an example of display according tothe first embodiment;

FIG. 10B is a diagram illustrating an example of display according tothe first embodiment;

FIG. 11 is a diagram illustrating an example of display according to asecond embodiment;

FIG. 12 is a diagram illustrating an example of display according to athird embodiment; and

FIG. 13 is a diagram illustrating an example of display according to afourth embodiment.

DETAILED DESCRIPTION

Embodiments of a medical information processing apparatus, an X-raydiagnostic apparatus, and a medical information processing program willhereinafter be described in detail while reference is made to thedrawings.

First Embodiment

With respect to a first embodiment, a medical information processingsystem 1 including a medical information processing apparatus 30 will bedescribed as an example. Furthermore, with respect to the firstembodiment, X-ray images acquired by an X-ray diagnostic apparatus 10will be described as an example of medical images. For example, themedical information processing system 1 includes, as illustrated in FIG.1, the X-ray diagnostic apparatus 10, an image storing apparatus 20, andthe medical information processing apparatus 30. FIG. 1 is a blockdiagram illustrating an example of a configuration of the medicalinformation processing system 1 according to the first embodiment.

As illustrated in FIG. 1, the X-ray diagnostic apparatus 10, the imagestoring apparatus 20, and the medical information processing apparatus30 are connected to one another via a network NW. The network NW may beformed of a closed local network in a hospital or may be a network viathe Internet. That is, the image storing apparatus 20 may be installedin the same facility as the X-ray diagnostic apparatus 10 and medicalinformation processing apparatus 30, or may be installed in a differentfacility. However, the medical information processing apparatus 30 istypically placed in an examination room where the X-ray diagnosticapparatus 10 is installed or in an operation room for operating theX-ray diagnostic apparatus 10.

For example, the X-ray diagnostic apparatus 10 sequentially acquiresX-ray images during a treatment procedure for a subject P and transmitsthe acquired X-ray images to the image storing apparatus 20 or themedical information processing apparatus 30. This X-ray diagnosticapparatus 10 will be described later.

The image storing apparatus 20 stores various types of medical images.For example, the image storing apparatus 20 receives and stores theX-ray images acquired by the X-ray diagnostic apparatus 10. The imagestoring apparatus 20 is a server of a picture archiving andcommunication system (PACS), for example.

The medical information processing apparatus 30 executes various typesof processing based on the X-ray images acquired from the X-raydiagnostic apparatus 10 or image storing apparatus 20. For example, themedical information processing apparatus 30 has, as illustrated in FIG.1, an input interface 31, a display 32, a memory 33, and processingcircuitry 34.

The input interface 31 receives various input operations from a user,converts the input operations received, into electric signals, andoutputs the electric signals to the processing circuitry 34. Forexample, the input interface 31 is implemented by any of: a mouse and akeyboard; a trackball; switches; buttons; a joystick; a touchpadenabling an input operation by a touch on an operation surface; a touchscreen having a display screen and a touchpad that have been integratedtogether; a non-contact input circuit using an optical sensor; and avoice input circuit. The input interface 31 may be formed of a tabletterminal that enables wireless communication with the medicalinformation processing apparatus 30, for example. Furthermore, the inputinterface 31 may be a circuit that receives an input operation from auser by motion capturing. For example, by processing signals acquiredvia a tracker or images acquired with respect to a user, the inputinterface 31 may receive a body motion or line of sight of the user asan input operation. In addition, the input interface 31 does notnecessarily include physical operating parts, such as a mouse and akeyboard. For example, examples of the input interface 31 include anelectric signal processing circuit that receives an electric signalcorresponding to an operation input from an external input deviceprovided separately from the medical information processing apparatus 30and outputs this electric signal to the processing circuitry 34.

The display 32 displays various types of information. For example, undercontrol of the processing circuitry 34, the display 32 displays X-rayimages acquired during a treatment procedure for the subject P. Displayof medical images on the display 32 will be described later.Furthermore, for example, the display 32 displays a graphical userinterface (GUI) for receiving various instructions and settings, from auser, via the input interface 31. For example, the display 32 is aliquid crystal display or a cathode ray tube (CRT) display. The display32 may be of the desktop type, or may be formed of, for example, atablet terminal that enables wireless communication with the medicalinformation processing apparatus 30.

The medical information processing apparatus 30 in FIG. 1 is describedherein as including the display 32, but the medical informationprocessing apparatus 30 may include, instead of or in addition to thedisplay 32, a projector. The projector may perform projection on, forexample, a screen, a wall, a floor, or a body surface of the subject P,under control of the processing circuitry 34. For example, the projectormay also perform projection on, for example, any plane, object, orspace, by projection mapping. In addition, the medical informationprocessing apparatus 30 may cause, instead of the display 32, or inaddition to the display 32, a display 108 of the X-ray diagnosticapparatus 10 to display an image to be displayed.

The memory 33 is implemented by, for example: a semiconductor memoryelement, such as a random access memory (RAM) or a flash memory; a harddisk; or an optical disk. For example, the memory 33 stores programs forthe circuitry included in the medical information processing apparatus30 to implement functions of the circuitry. Furthermore, the memory 33stores X-ray images acquired from the X-ray diagnostic apparatus 10 orimage storing apparatus 20. The memory 33 may be implemented by a servergroup (a cloud) connected to the medical information processingapparatus 30 via the network NW.

The processing circuitry 34 controls the overall operation of themedical information processing apparatus 30 by executing a controllingfunction 34 a, an image acquiring function 34 b, an event acquiringfunction 34 c, a managing function 34 d, and an outputting function 34e. The image acquiring function 34 b is an example of an image acquiringunit. Furthermore, the event acquiring function 34 c is an example of anevent acquiring unit. The managing function 34 d is an example of amanaging unit. The outputting function 34 e is an example of an outputunit.

For example, by reading and executing a program corresponding to thecontrolling function 34 a from the memory 33, the processing circuitry34 controls various functions including the image acquiring function 34b, the event acquiring function 34 c, the managing function 34 d, andthe outputting function 34 e, on the basis of various input operationsreceived from a user via the input interface 31.

Furthermore, for example, by reading and executing a programcorresponding to the image acquiring function 34 b, the processingcircuitry 34 sequentially acquires X-ray images during a treatmentprocedure for the subject P. In addition, for example, by reading andexecuting a program corresponding to the event acquiring function 34 c,the processing circuitry 34 sequentially acquires events in thetreatment procedure on the basis of the X-ray images, during thetreatment procedure. Furthermore, for example, by reading and executinga program corresponding to the managing function 34 d, the processingcircuitry 34 manages the X-ray images and events, in association withtemporal information on the treatment procedure. In addition, forexample, by reading and executing a program corresponding to theoutputting function 34 e, the processing circuitry 34 outputs the X-rayimages such that relations between the X-ray images and the events areable to be known. Processing by the image acquiring function 34 b, theevent acquiring function 34 c, the managing function 34 d, and theoutputting function 34 e will be described later.

The processing functions are stored in the memory 33 in the medicalinformation processing apparatus 30 illustrated in FIG. 1, theprocessing functions being in the form of programs executable by acomputer. The processing circuitry 34 is a processor that implements thefunctions corresponding to the programs by reading and executing theprograms from the memory 33. In other words, the processing circuitry 34that has read the programs has the functions corresponding to the readprograms.

The controlling function 34 a, the image acquiring function 34 b, theevent acquiring function 34 c, the managing function 34 d, and theoutputting function 34 e have been described as being implemented by thesingle processing circuitry 34 in FIG. 1, but the processing circuitry34 may be formed of a combination of plural independent processors andeach of these processors may implement a function by executing aprogram. Furthermore, any of the processing functions that theprocessing circuitry 34 has may be implemented by being distributed toplural pieces of processing circuitry or integrated into singleprocessing circuitry, as appropriate.

In addition, the processing circuitry 34 may implement a function byusing a processor of an external device connected via the network NW.For example, by reading and executing the programs corresponding to therespective functions from the memory 33 and using, as a computationalresource, a server group (a cloud) connected to the medical informationprocessing apparatus 30 via the network NW, the processing circuitry 34implements the functions illustrated in FIG. 1.

The X-ray diagnostic apparatus 10 will be described next, using FIG. 2.FIG. 2 is a block diagram illustrating an example of a configuration ofthe X-ray diagnostic apparatus 10 according to the first embodiment. Asillustrated in FIG. 2, the X-ray diagnostic apparatus 10 includes anX-ray high voltage generator 101, an X-ray tube 102, an X-ray limitingdevice 103, a tabletop 104, a C-arm 105, an X-ray detector 106, a memory107, the display 108, an input interface 109, and processing circuitry110.

Under control of the processing circuitry 110, the X-ray high voltagegenerator 101 supplies high voltage to the X-ray tube 102. For example,the X-ray high voltage generator 101 has: a high voltage generator thathas electric circuitry including a transformer and a rectifier andgenerates high voltage to be applied to the X-ray tube 102; and an X-raycontroller that controls output voltage according to X-rays to beemitted by the X-ray tube 102. The high voltage generator may be of thetransformer-type or the inverter-type.

The X-ray tube 102 is a vacuum tube having: a cathode (a filament) thatgenerates thermions; and an anode (a target) that receives collision ofthe thermions and generates X-rays. The X-ray tube 102 generates X-raysby emitting the thermions from the cathode to the anode using the highvoltage supplied from the X-ray high voltage generator 101.

The X-ray limiting device 103 has a collimator that narrows the range tobe irradiated with the X-rays generated by the X-ray tube 102, and afilter that adjusts the X-rays emitted from the X-ray tube 102.

The collimator in the X-ray limiting device 103 has, for example, fourdiaphragm blades that are able to be slid. By sliding the diaphragmblades, the collimator narrows the X-rays generated by the X-ray tube102 and causes the subject P to be irradiated with the X-rays that havebeen narrowed. The diaphragm blades are plate like members formed oflead, for example, and are provided near an X-ray emission port of theX-ray tube 102, for adjustment of the range to be irradiated with theX-rays.

To reduce the dose of radiation received by the subject P and improvethe image quality of X-ray image data, the filter in the X-ray limitingdevice 103 reduces the soft X-ray components that are easily absorbedinto the subject P and/or reduces high energy components that causereduction in contrast of the X-ray image data, by changing radiationquality of transmitted X-rays by means of the material and/or thicknessof the filter. Furthermore, the filter changes the dose of X-rays andthe range to be irradiated by means of the material, thickness, and/orposition of the filter, for example, to attenuate the X-rays such thatthe X-rays emitted from the X-ray tube 102 to the subject P have apredetermined distribution.

For example, the X-ray limiting device 103 has a driving systemincluding a motor and an actuator, and controls, under control of theprocessing circuitry 110 described later, emission of X-rays byoperating the driving system. For example, the X-ray limiting device 103adjusts apertures of the diaphragm blades of the collimator to controlthe range to be irradiated with the X-rays emitted to the subject P, byapplying driving voltage to the driving system according to a controlsignal received from the processing circuitry 110. Furthermore, forexample, the X-ray limiting device 103 adjusts the position of thefilter to control the distribution of doses of X-rays emitted to thesubject P, by applying driving voltage to the driving system accordingto a control signal received from the processing circuitry 110.

The tabletop 104 is a bed on which the subject P is laid and thetabletop 104 is placed on a bed unit not illustrated in the drawings.The subject P is not included in the X-ray diagnostic apparatus 10. Forexample, the bed unit has a driving system including a motor and anactuator, and controls movement and inclination of the tabletop 104 byoperating the driving system under control of the processing circuitry110 described later. The bed unit moves and inclines the tabletop 104,for example, by applying driving voltage to the driving system accordingto a control signal received from the processing circuitry 110.

The C-arm 105 holds the X-ray tube 102 and X-ray limiting device 103opposite to the X-ray detector 106, with the subject P interposedtherebetween. For example, the C-arm 105 has a driving system includinga motor and an actuator, and rotates and moves by operation of thedriving system under control of the processing circuitry 110 describedlater. For example, the C-arm 105 rotates and moves the X-ray tube 102and X-ray limiting device 103 and the X-ray detector 106, relatively tothe subject P to control the X-ray irradiation position and irradiationangle, by applying driving voltage to the driving system according to acontrol signal received from the processing circuitry 110. The X-raydiagnostic apparatus 10 in FIG. 2 is described herein as a single-planeapparatus as an example, but the embodiment is not limited to thisexample and the X-ray diagnostic apparatus 10 may be a biplaneapparatus.

The X-ray detector 106 is, for example, an X-ray flat panel detector(FPD) having detecting elements arranged in a matrix. The X-ray detector106 detects X-rays emitted from the X-ray tube 102 and transmittedthrough the subject P, and outputs a detection signal corresponding to adose of X-rays detected, to the processing circuitry 110. The X-raydetector 106 may be an indirect conversion detector having a grid, ascintillator array, and an optical sensor array, or a direct conversiondetector having a semiconductor element that converts X-rays incidentthereon into an electric signal.

The memory 107 is implemented by, for example: a semiconductor memoryelement, such as a RAM or a flash memory; a hard disk; or an opticaldisk. For example, the memory 107 stores programs corresponding tovarious functions read and executed by the processing circuitry 110. Thememory 107 may be implemented by a cloud.

The display 108 displays various types of information. For example,under control of the processing circuitry 110, the display 108 displaysa GUI for receiving a user's instructions and various X-ray images. Forexample, the display 108 is a liquid crystal display or a CRT display.The display 108 may be of the desktop type or may be formed of, forexample, a tablet terminal enabling wireless communication with theprocessing circuitry 110.

The X-ray diagnostic apparatus 10 in FIG. 2 is described herein asincluding the display 108 but the X-ray diagnostic apparatus 10 mayinclude, instead of, or in addition to the display 108, a projector. Theprojector may perform projection on, for example, a screen, a wall, afloor, or a body surface of the subject P, under control of theprocessing circuitry 110. For example, the projector may performprojection on any plane, object, or space, by projection mapping.

The input interface 109 receives various input operations from a user,converts the input operations received, into electric signals, andoutputs the electric signals to the processing circuitry 110. Forexample, the input interface 109 is implemented by any of: a mouse and akeyboard; a trackball; switches; buttons; a joystick; a touchpadenabling an input operation by a touch on an operation surface; a touchscreen having a display screen and a touchpad that have been integratedtogether; a non-contact input circuit using an optical sensor; and avoice input circuit. The input interface 109 may be formed of, forexample, a tablet terminal that enables wireless communication with theprocessing circuitry 110. Furthermore, the input interface 109 may be acircuit that receives an input operation from a user by motioncapturing. For example, by processing signals acquired via a tracker orimages acquired with respect to a user, the input interface 109 mayreceive a body motion or line of sight of the user as an inputoperation. In addition, the input interface 109 does not necessarilyinclude physical operating parts, such as a mouse and a keyboard. Forexample, examples of the input interface 109 include an electric signalprocessing circuit that receives an electric signal corresponding to anoperation input from an external input device provided separately fromthe X-ray diagnostic apparatus 10 and outputs this electric signal tothe processing circuitry 110.

The processing circuitry 110 controls the overall operation of the X-raydiagnostic apparatus 10 by executing a controlling function 110 a, anacquiring function 110 b, and an outputting function 110 c.

For example, by reading and executing a program corresponding to thecontrolling function 110 a from the memory 107, the processing circuitry110 controls various functions, such as the acquiring function 110 b andoutputting function 110 c, on the basis of various input operationsreceived from a user via the input interface 109.

For example, by reading and executing a program corresponding to theacquiring function 110 b from the memory 107, the processing circuitry110 acquires X-ray images from the subject P. For example, the acquiringfunction 110 b sequentially acquires X-ray images on the basis of X-raystransmitted through the subject P during a treatment procedure for thesubject P.

For example, by controlling the X-ray high voltage generator 101 toadjust voltage to be supplied to the X-ray tube 102, the acquiringfunction 110 b controls the dose of X-rays to be emitted to the subjectP and performs on-off control. Furthermore, by controlling operation ofthe X-ray limiting device 103 to adjust the diaphragm blades that thecollimator has, the acquiring function 110 b controls the range to beirradiated with X-rays emitted to the subject P. In addition, bycontrolling operation of the X-ray limiting device 103 to adjust theposition of the filter, the acquiring function 110 b controls thedistribution of doses of X-rays. Furthermore, by controlling operationof the C-arm 105 to change the position and angle of the C-arm 105relatively to the subject P, the acquiring function 110 b controls theimaging position and imaging angle. The position and angle of the C-arm105 are also referred to herein as arm position information. Inaddition, the acquiring function 110 b generates X-ray images on thebasis of detection signals received from the X-ray detector 106 andstores the generated X-ray images into the memory 107.

Furthermore, by reading and executing a program corresponding to theoutputting function 110 c from the memory 107, the processing circuitry110 controls transmission and reception of data via the network NW anddisplay at the display 32. For example, the outputting function 110 ctransmits the X-ray images acquired by the acquiring function 110 b tothe medical information processing apparatus 30 via the network NW andcauses the display 32 to display the X-ray images. In addition, forexample, the outputting function 110 c causes the display 32 to displaya GUI for receiving input operations from a user.

The processing functions have been stored in the memory 107 of the X-raydiagnostic apparatus 10 illustrated in FIG. 2, the processing functionsbeing in the form of programs executable by a computer. The processingcircuitry 110 is a processor that implements functions corresponding tothe programs by reading and executing the programs from the memory 107.In other words, the processing circuitry 110 that has read the programshas the functions corresponding to the read programs.

The controlling function 110 a, the acquiring function 110 b, and theoutputting function 110 c have been described as being implemented bythe single processing circuitry 110 in FIG. 2, but the processingcircuitry 110 may be formed of a combination of plural independentprocessors and the functions may be implemented by the processorsexecuting the programs. Furthermore, any of the processing functionsthat the processing circuitry 110 has may be implemented by beingdistributed to plural pieces of processing circuitry or integrated intosingle processing circuitry.

In addition, the processing circuitry 110 may implement a function byusing a processor of an external device connected via the network NW.For example, by reading and executing the programs corresponding to therespective functions from the memory 107 and using, as a computationalresource, a server group (a cloud) connected to the X-ray diagnosticapparatus 10 via the network NW, the processing circuitry 110 implementsthe functions illustrated in FIG. 2.

The above description is on the medical information processing system 1including the X-ray diagnostic apparatus 10, the image storing apparatus20, and the medical information processing apparatus 30. The followingdescription is on processing performed in the medical informationprocessing system 1.

For example, the X-ray diagnostic apparatus 10 sequentially acquiresX-ray images from the subject P in a treatment procedure for the subjectP. Furthermore, the medical information processing apparatus 30sequentially acquires the X-ray images from the X-ray diagnosticapparatus 10 and causes the display 32 to sequentially display theacquired X-ray images in real time. A user is thereby able to check, forexample, a structure in the body of the subject P and a medical deviceinserted in the body of the subject P, and also able to proceed smoothlywith the treatment procedure. That is, by acquiring and displaying X-rayimages, the medical information processing system 1 is able to support auser (a medical doctor in attendance) who conducts a treatmentprocedure.

A treatment procedure is often executed by a team of people including amedical doctor in attendance, and a member other than the medical doctorin attendance may want to know the current situation in the treatmentprocedure by referring to real-time X-ray images. Furthermore, duringexecution of a treatment procedure, a medical doctor in attendance oranother member may refer to an X-ray image that has been acquired in thetreatment procedure.

However, characteristics specific to the current situation in atreatment procedure may not necessarily be represented by real-timeX-ray images and the process leading to the current situation may bedifficult to be determined just from the real-time X-ray images.Therefore, even if a member in a team refers to real-time X-ray imagesin a treatment procedure, the member may be unable to know the currentsituation in the treatment procedure and communication in the team maythus be made difficult. Furthermore, for example, in a case where amember in a team attempts to display an X-ray image that has beenacquired in a treatment procedure on the display 108 as a referenceimage during execution of the treatment procedure, finding the imageneeded may be troublesome.

The medical information processing apparatus 30 thus enables effectiveutilization of medical images, such as X-ray images, by means ofprocessing by the processing circuitry 34. The processing executed bythe processing circuitry 34 will be described in detail below.

Firstly, the image acquiring function 34 b sequentially acquires X-rayimages during a treatment procedure for the subject P. The imageacquiring function 34 b may acquire the X-ray images via the imagestoring apparatus 20, or may acquire the X-ray images directly from theX-ray diagnostic apparatus 10. Furthermore, the event acquiring function34 c sequentially acquires events in the treatment procedure during thetreatment procedure.

Examples of an event acquiring process by the event acquiring function34 c will now be described using FIG. 3A, FIG. 3B, and FIG. 3C. FIG. 3A,FIG. 3B, and FIG. 3C are diagrams illustrating examples of an eventacquiring process according to the first embodiment.

FIG. 3A will be described firstly. In the case illustrated in FIG. 3A,before a treatment procedure is starred, the event acquiring function 34c receives input of patient information (Step S101) and receives aselection of a procedure (Step S102). For example, the event acquiringfunction 34 c may execute Step S101 and Step S102 by receiving an inputoperation from a user via the input interface 31. Step S101 and StepS102 may be omitted as appropriate.

Furthermore, the event acquiring function 34 c receives a selection ofan event flow (Step S103). A case where an event flow including an eventE1, an event E2, and an event E3 has been selected will be described asan example with respect to FIG. 3A. For example, the event acquiringfunction 34 c may execute Step S103 by receiving an input operation froma user via the input interface 31. The memory 33 may store a databaseincluding plural types of event flows beforehand, and the eventacquiring function 34 c may receive an operation from a user, theoperation being for selecting any one of the event flows from thatdatabase, for example. Or, on the basis of the patient informationreceived at Step S101 and/or the procedure selected at Step S102, theevent acquiring function 34 c may automatically select an event flow.

After Step S103, the event acquiring function 34 c determines that thetreatment procedure is in a situation of the first event E1 (Step S104).Furthermore, after the treatment procedure is started, the imageacquiring function 34 b acquires X-ray images acquired at the X-raydiagnostic apparatus 10 (Step S105). The event acquiring function 34 cmay acquire the event E1 for the X-ray images acquired at Step S105.

Furthermore, the managing function 34 d manages the X-ray imagesacquired at Step S105 and the event E1, in association with temporalinformation on the treatment procedure. For example, the managingfunction 34 d stores the X-ray images acquired at Step S105 withsupplementary information added to the X-ray images, the supplementaryinformation being the event E1 and the temporal information. Themanaging function 34 d stores the event E1 and the temporal informationin Digital Imaging and Communications in Medicine (DICOM) tags of theX-ray images. The temporal information may be information on dates andtimes on and at which the X-ray images were acquired at the X-raydiagnostic apparatus 10, or may be information on dates and times on andat which the medical information processing apparatus 30 acquired theX-ray images from the X-ray diagnostic apparatus 10. Furthermore, themanaging function 34 d may further store, as the supplementaryinformation on the X-ray images, the patient information received atStep S101 or the procedure selected at Step S102.

Subsequently, the event acquiring function 34 c determines whether ornot the event E1 has ended (Step S106) and if the event E1 has not ended(No at Step S106), the event acquiring function 34 c proceeds to StepS105 again. For example, on the basis of the X-ray images acquired, theevent acquiring function 34 c automatically determines whether or notthe event E1 has ended. If, for example, the event E1 is an event where“a medical device is inserted from the femoral region of the subject Pand caused to reach the heart that is a region to be treated”, the eventacquiring function 34 c may automatically determine whether or not theevent E1 has ended on the basis of a positional relation between themedical device and the heart of the subject P that appear in an X-rayimage. If the event E1 has not ended, the event acquiring function 34 cproceeds to Step S105 again and acquires the event E1 for X-ray imagesnewly acquired. That is, the event acquiring function 34 c may acquirean event on the basis of X-ray images. Therefore, even if the X-rayimages themselves do not represent characteristics specific to the eventE1, information on the event E1 may be assigned to the X-ray images.

Or, by comparing positional information on a peripheral device with adatabase, the event acquiring function 34 c may automatically determinewhether or not the event E1 has ended. For example, if the C-arm 105 ismoved to follow the position of the distal end of the medical device,the event acquiring function 34 c may automatically determine whether ornot the event E1 has ended on the basis of positional information on theC-arm 105. Or, by receiving an input operation from a user via the inputinterface 31, the event acquiring function 34 c may determine whether ornot the event E1 has ended.

On the contrary, if the event E1 has ended (Yes at Step S106), the eventacquiring function 34 c determines that the treatment procedure hasproceeded to be in a situation of the event E2 (Step S107). Furthermore,the image acquiring function 34 b acquires X-ray images acquired at theX-ray diagnostic apparatus 10 (Step S108). The event acquiring function34 c may acquire the event E2 for the X-ray images acquired at StepS108. For example, the event acquiring function 34 c determines whetheror not the event E1 has ended on the basis of the X-ray images acquiredat Step S105 and if the event E1 has ended, the event acquiring function34 c determines that the treatment procedure has proceeded to be in thesituation of the event E2 and acquires the event E2 for the X-ray imagesnewly acquired at Step S108. Furthermore, the managing function 34 dmanages the X-ray images acquired at Step S108 and the event E2, inassociation with temporal information on the treatment procedure. Thedetermination of the end of the event E1 at Step S106 may be executed bydetecting characteristics specific to the end of the event E1 or may beexecuted by determining start of the event E2 through detection ofcharacteristics specific to the event E2. This method of determinationis similarly applicable to Step S109 and so on.

Subsequently, the event acquiring function 34 c determines whether ornot the event E2 has ended (Step S109) and if the event E2 has not ended(No at Step S109), the event acquiring function 34 c proceeds to StepS108 again. For example, the event acquiring function 34 c determineswhether or not the event E2 has ended on the basis of the X-ray images,and if the event E2 has not ended, the event acquiring function 34 cproceeds to Step S108 again and acquires the event E2 for X-ray imagesnewly acquired. On the contrary, if the event E2 has ended (Yes at StepS109), the event acquiring function 34 c determines that the treatmentprocedure has proceeded to be in a situation of the event E3 (StepS110). Furthermore, the image acquiring function 34 b acquires X-rayimages acquired at the X-ray diagnostic apparatus 10 (Step S111). Theevent acquiring function 34 c may acquire the event E3 for the X-rayimages acquired at Step S111. For example, the event acquiring function34 c determines whether or not the event E2 has ended on the basis ofthe X-ray images acquired at Step S108, and if the event E2 has ended,the event acquiring function 34 c determines that the treatmentprocedure has proceeded to be in the situation of the event E3 andacquires the event E3 for the X-ray images newly acquired at Step S111.Furthermore, the managing function 34 d manages the X-ray imagesacquired at Step S111 and the event E3, in association with temporalinformation on the treatment procedure.

Subsequently, the event acquiring function 34 c determines whether ornot the event E3 has ended (Step S112) and if the event E3 has not ended(No at Step S112), the event acquiring function 34 c proceeds to StepS111 again. For example, the event acquiring function 34 c determineswhether or not the event E3 has ended on the basis of the X-ray images,and if the event E3 has not ended, the event acquiring function 34 cproceeds to Step S111 again and acquires the event E3 for X-ray imagesnewly acquired. On the contrary, if the event E3 has ended (Yes at StepS112), the event acquiring function 34 c ends the process.

That is, according to the process illustrated in FIG. 3A, the imageacquiring function 34 b sequentially acquires X-ray images during atreatment procedure for the subject P and the event acquiring function34 c sequentially acquires events in the treatment procedure during thetreatment procedure. That is, on the basis of the X-ray imagessequentially acquired, the event acquiring function 34 c sequentiallyidentifies images in which the events occurred, from the X-ray imagessequentially acquired. For example, by determining whether or not eachevent included in an event flow has ended on the basis of the X-rayimages, the event acquiring function 34 c may sequentially acquireevents for X-ray images newly acquired. Furthermore, the managingfunction 34 d may manage the X-ray images and events, in associationwith temporal information on the treatment procedure. That is, themanaging function 34 d stores medical images acquired and also storesinformation on events that occurred, in association with imagesidentified as images in which the events occurred, the images being fromthe medical images stored.

FIG. 3B will be described next. In the case illustrated in FIG. 3B,before a treatment procedure is started, the event acquiring function 34c receives input of patient information (Step S201) and receives aselection of a procedure (Step S202). For example, the event acquiringfunction 34 c may execute Step S201 and Step S202 by receiving an inputoperation from a user via the input interface 31. Step S201 and StepS202 may be omitted as appropriate.

Furthermore, the event acquiring function 34 c sets an event name. Forexample, after Step S202, the event acquiring function 34 c sets thefirst event name for the treatment procedure (Step S203). The eventacquiring function 34 c may set the event name by receiving an inputoperation from a user via the input interface 31, for example. Forexample, the event acquiring function 34 c may set the event name byvoice input. Or, on the basis of the patient information received atStep S201 and/or the procedure selected at Step S202, the eventacquiring function 34 c may automatically set the event name.

Subsequently, the event acquiring function 34 c determines whether ornot the event name is to be corrected (Step S204). For example, in acase where a plan has been changed or automatic recognition and a user'srecognition are in disagreement, the event acquiring function 34 c maydetermine that the event name is to be corrected (Yes at Step S204) andcorrect the event name (Step S205). The event acquiring function 34 cmay correct the event name by receiving an input operation from a useror automatically reset the event name.

On the contrary, if the event name is not to be corrected (No at StepS204), the image acquiring function 34 b acquires X-ray images acquiredat the X-ray diagnostic apparatus 10 (Step S206). The event acquiringfunction 34 c may acquire the event name set at Step S203 or correctedat Step S205, for the X-ray images acquired at Step S206. Furthermore,the managing function 34 d may manage the X-ray images and event, inassociation with temporal information on the treatment procedure.

Subsequently, the event acquiring function 34 c determines whether ornot the event having the event name set at Step S203 or corrected atStep S205 has ended (Step S207), and if the event has not ended (No atStep S207), the event acquiring function 34 c proceeds to Step S206again. For example, on the basis of the X-ray images acquired, the eventacquiring function 34 c automatically determines whether or not theevent has ended. If the event has not ended, the event acquiringfunction 34 c proceeds to Step S206 again and acquires the event havingthe event name set at Step S203 or corrected at Step S205 for X-rayimages newly acquired. That is, the event acquiring function 34 c mayacquire an event on the basis of X-ray images.

Or, at Step S207, by comparing positional information on a peripheraldevice with a database, the event acquiring function 34 c mayautomatically determine whether or not the event has ended. Furthermore,by receiving an input operation from a user via the input interface 31,the event acquiring function 34 c may determine whether or not the eventhas ended.

On the contrary, if the event has ended (Yes at Step S207), the event,acquiring function 34 c determines whether or not there is anysubsequent event (Step S208). If there is any subsequent event (Yes atStep S208), the event acquiring function 34 c proceeds to Step S203again and sets an event name for the subsequent event. On the contrary,if there is no subsequent event (No at Step S208), the event acquiringfunction 34 c ends the process.

That is, according to the process illustrated in FIG. 3B, the imageacquiring function 34 b sequentially acquires X-ray images during atreatment procedure for the subject P and the event acquiring function34 c sequentially acquires events in the treatment procedure during thetreatment procedure. For example, by determining whether or not an eventhaving an event name set at Step S203 or corrected at Step S205 hasended on the basis of X-ray images, the event acquiring function 34 cmay sequentially acquire events for X-ray images newly acquired.Furthermore, the managing function 34 d may manage the X-ray images andevents, in association with temporal information on the treatmentprocedure.

FIG. 3C will be described next. In the case illustrated in FIG. 3C,before a treatment procedure is started, the event acquiring function 34c receives input of patient information (Step S301) and receives aselection of a procedure (Step S302). For example, the event acquiringfunction 34 c may execute Step S301 and Step S302 by receiving an inputoperation from a user via the input interface 31. Step S301 and StepS302 may be omitted as appropriate.

Subsequently, the image acquiring function 34 b acquires X-ray imagesacquired at the X-ray diagnostic apparatus 10 (Step S303). Next, theevent acquiring function 34 c sets an event name by a process using theX-ray images acquired at Step S303 (Step S304). For example, the eventacquiring function 34 c detects a medical device from the X-ray imagesacquired at Step S303 and identifies the type of the medical devicedetected and the position of the medical device in the body of thesubject P. In a case where a balloon used to dilate a stenotic portionof a blood vessel is detected as a medical device and dilation of thatballoon is detected in each of consecutive frames, for example, theevent acquiring function 34 c may acquire an event, “Balloon dilation”.That is, the event acquiring function 34 c may acquire an event on thebasis of X-ray images. Furthermore, the managing function 34 d maymanage the X-ray images and event, in association with temporalinformation on the treatment procedure.

Subsequently, the event acquiring function 34 c determines whether ornot there is any subsequent X-ray image (Step S305) and if there is anysubsequent X-ray image (Yes at Step S305), the event acquiring function34 c proceeds to Step S303 again. On the contrary, if there is nosubsequent X-ray image (No at Step S305), the event acquiring function34 c ends the process. That is, according to the process illustrated inFIG. 3C, the image acquiring function 34 b may sequentially acquireX-ray images during a treatment procedure for the subject P, the eventacquiring function 34 c may sequentially acquire events in the treatmentprocedure during the treatment procedure on the basis of the X-rayimages, and the managing function 34 d may manage the X-ray images andevent, in association with temporal information on the treatmentprocedure.

FIG. 3A, FIG. 3B, and FIG. 3C are just examples, and the event acquiringfunction 34 c may acquire events by any of various methods. For example,the event acquiring function 34 c may acquire events on the basis ofimages acquired by a medical diagnostic imaging device (a modality)different from the X-ray diagnostic apparatus 10. For example, at StepS304 in FIG. 3C, the event acquiring function 34 c may set an event nameon the basis of ultrasound images acquired in parallel with the X-rayimages.

As illustrated in FIG. 4, for example, the event acquiring function 34 cmay execute any of the various event acquiring methods described above,on the basis of a database storing event types. This database is managedin the memory 33, for example, and may be read therefrom as appropriate.For example, the event acquiring function 34 c compares any of varioustypes of information with event names registered in the database andacquires any matching or similar event name, the various types ofinformation including control information on a holding device or bedunit, input information from an input terminal, input information from amicrophone or a camera, and ultrasound images acquired by intravascularultrasonography (IVUS). The holding device in FIG. 4 is, for example,the C-arm 105 illustrated in FIG. 2. Furthermore, the bed unit in FIG. 4is, for example, a device that controls movement and inclination of thetabletop 104 illustrated in FIG. 2. Furthermore, the input terminal inFIG. 4 is an example of the input interface 31 illustrated in FIG. 1 andis, for example, a mouse and a keyboard. In addition, the microphone andcamera in FIG. 4 are an example of the input interface 31 illustrated inFIG. 1. Furthermore, the ultrasound images by IVUS are managed in thememory 33, for example, and may be read therefrom as appropriate. Thatis, by registering event names in the database beforehand and acquiringan event using the database, the event acquiring function 34 c acquiresthe same event name for events of the same type and is thus able tomanage events efficiently. FIG. 4 is a diagram illustrating an exampleof the event acquiring process according to the first embodiment.

Next, the outputting function 34 e outputs X-ray images such thatrelations between the X-ray images and events are able to be known. Forexample, on the basis of temporal information associated with the X-rayimages and events, the outputting function 34 e displays a timelinehaving the events arranged chronologically, on the display 32.Furthermore, the outputting function 34 e displays the X-ray images onthe display 32 such that chronological relations between the X-rayimages and the events displayed in the timeline are able to be known.

For example, the outputting function 34 e displays a display screenincluding an area R11, an area R12, and an area R13, as illustrated inFIG. 5, on the display 32. The outputting function 34 e displays thedisplay screen in FIG. 5 on the display 32 during a treatment procedure,for example. FIG. 5 is a diagram illustrating an example of displayaccording to the first embodiment.

Specifically, the outputting function 34 e displays a timeline havingevent arranged chronologically, in the area R11. FIG. 5 illustrates acase where, for example, an event E11 (Arrival at target region), anevent E12 (L-ICM balloon contrast imaging), and an event E13 (Stentplacement) are displayed in a timeline. Furthermore, FIG. 5 illustratesthat the event E12 is formed of detailed events, such as an event E121(Checking stenotic portion (injection of contrast agent)), an event E122(Dilation of balloon), and an event E123 (Checking stenotic portion(injection of contrast agent)).

In addition, the outputting function 34 e displays the current eventwith a line L1 in the area R11. That is, in FIG. 5, the event E12including the event E121 is the current event, the event E11 is a pastevent, and the event E13 is a subsequent event. Furthermore, theoutputting function 34 e may display X-ray images associated with theevent E121 in the area R12 or the area R13. For example, the outputtingfunction 34 e displays, as a thumbnail, an X-ray image I11 acquiredwhile the event E121 was occurring, in the area R12 and also displaysthe X-ray image I11 enlarged in the area R13. That is, by displaying theline L1 on the timeline, the outputting function 34 e may display theX-ray image I11 such that chronological relations between the X-rayimage I11 and the events displayed in the timeline are able to be known.For example, by referring to the display in FIG. 5, members of a teamfor a treatment procedure are able to know the current situation of thetreatment procedure and readily communicate with each other.

The outputting function 34 e may display all of the events included inthe treatment procedure or may display only some of the events. Forexample, the outputting function 34 e may select events to be displayedon the basis of a list of events to be displayed, and not display anyevent not included in this list. The outputting function 34 e is therebyable to reduce the number of events to be displayed and prevent a userfrom getting confused by display of too much information.

Furthermore, the line L1 is described herein to indicate the currentevent, but the line L1 may be movable on the timeline in any way. Forexample, during a treatment procedure, a user may move the line L1 tothe position of the event E11 that is a past event. In this case, theoutputting function 34 e displays, on the display 32, X-ray imagesassociated with the event E11 at which the line L1 that has been movedis positioned. Candidates are thereby able to be narrowed down usinginformation on a situation in a treatment procedure in a case whereX-ray images that have been acquired are displayed as reference imagesduring execution of the treatment procedure, and the trouble to searchfor images needed is thus able to be reduced.

The outputting function 34 e may further display, in addition to anX-ray image associated with an event at which the line L1 is positioned,a related image that is related to that X-ray image. For example, in acase where the line L1 is positioned at the event E121 as illustrated inFIG. 5, the outputting function 34 e may display, in addition to theX-ray image I11 associated with the event E121, related images, such asan image I12, an image I13, and an image I14, in the area R12.Furthermore, in a case where any of these related images is selected bya user, the outputting function 34 e may display that related imageenlarged in the area R13.

The related images related to the X-ray image I11 are, for example,images having supplementary information that is the same as or similarto that of the X-ray image I11. For example, the outputting function 34e retrieves any medical image having supplementary information, such aspatient information or an event, from medical images stored in thememory 33 or image storing apparatus 20, the supplementary informationbeing similar to that of the X-ray image I11, and displays that medicalimage as a related image on the display 32. The outputting function 34 edisplays, for example, an X-ray image acquired from the subject P in theevent E11 that is a past event, as a related image related to the X-rayimage I11, on the display 32.

Furthermore, for example, a related image related to the X-ray image I11is an image having image features that are similar to those of the X-rayimage I11. The outputting function 34 e retrieves an image having amedical device appearing in the image, the medical device being the sameas or similar to that in the X-ray image I11 or an image acquired for anorgan that is the same as that in the X-ray image I11, from medicalimages stored in the memory 33 or image storing apparatus 20, anddisplays that image as a related image on the display 32, for example.

Furthermore, for example, a related image related to the X-ray image I11is an X-ray image having arm position information that is the same as orsimilar to that of the X-ray image I11. That is, the outputting function34 e retrieves an X-ray image having an imaging position and an imagingangle that are the same as or similar to those of the X-ray image I11from medical images stored in the memory 33 or image storing apparatus20, and displays that X-ray image as a related image, on the display 32.The outputting function 34 e is thereby able to narrow down relatedimages to be displayed to those near the real-time arm angle, in a casewhere display is performed during a treatment procedure, for example.

In a case where a related image is selected on the basis of arm positioninformation, any X-ray image having dissimilar arm position informationis excluded. The outputting function 34 e may notify a user of the factthat there is an X-ray image for which display has been omitted becauseof its dissimilar arm position information. For example, the outputtingfunction 34 e may display text on the timeline, the text indicating thatthere is an X-ray image for which display has been omitted because ofits dissimilar arm position information.

Furthermore, a related image related to the X-ray image I11 may be aprocessed image generated on the basis of the X-ray image I11. Forexample, the outputting function 34 e may retrieve a parametric imaging(PI) image generated on the basis of the X-ray image I11, from medicalimages stored in the memory 33 or image storing apparatus 20, anddisplay, on the display 32, the PI image as a related image related tothe X-ray image I11. The PI image is a color image generated bycalculating a parameter, such as a bloodstream arrival time from X-rayimages corresponding to plural frames resulting from contrast imaging ofa blood vessel and assigning a color according to a value of theparameter to each pixel.

The outputting function 34 e may read a processed image that has beengenerated or read an application that has been used to generate aprocessed image. For example, the outputting function 34 e may read a PIimage stored in the memory 33 or image storing apparatus 20 or read anapplication used to generate a PI image. In the case where theoutputting function 34 e reads the application, the outputting function34 e may generate the PI image from X-ray images corresponding to pluralframes including the X-ray image I11 and display the PI image as arelated image on the display 32, for example.

Furthermore, the outputting function 34 e may process or generate anddisplay a related image according to the X-ray image I11. For example,in a case where the outputting function 34 e has read a PI image basedon the X-ray image I11, the outputting function 34 e may display the PIimage as a related image after processing the display mode, such as thewindow level (WL) or window width (WW), of the PI image to facilitatecomparison between the X-ray image I11 and the PI image. Furthermore,for example, in a case where the outputting function 34 e has read anapplication used to generate a PI image, the outputting function 34 emay generate the PI image by setting the WL or WW to facilitatecomparison between the X-ray image I11 and the PI image and display thePI image as a related image.

Although PI images have been described as an example of processed imagesgenerated on the basis of the X-ray image I11, the embodiment is notlimited to this example. For example, the outputting function 34 e maydisplay, as a related image related to the X-ray image I11, a bloodvessel wall image having only a blood vessel wall extracted from theX-ray image I11, or a sketch image having a guide line marked on theX-ray image I11, on the display 32. Furthermore, for example, theoutputting function 34 e may display, as a related image related to theX-ray image I11, an image having a blood vessel wall image or sketchimage and the X-ray image III superimposed on each other, the bloodvessel wall image or sketch image having been generated on the basis ofan X-ray image acquired before the X-ray image I11, on the display 32.

The outputting function 34 e may display a processed image near theoriginal image. For example, the outputting function 34 e may display aprocessed image generated on the basis of the X-ray image I11 at aposition adjacent to the X-ray image I11 on a timeline. Or, theoutputting function 34 e may display a processed image at a positioncorresponding to the date and time on and at which the processed imagewas generated, on a timeline.

Furthermore, a related image related to the X-ray image I11 is notnecessarily an X-ray image, and may be an image acquired using anapparatus different from the X-ray diagnostic apparatus 10. For example,the outputting function 34 e may display, as a related image related tothe X-ray image I11, an ultrasound image acquired while the event E121was occurring, on the display 32. Furthermore, for example, theoutputting function 34 e may display, as a related image, an ultrasoundimage acquired in the past from the subject P, on the display 32. Theoutputting function 34 e may display, as a related image, an IVUS imageor transesophageal echocardiography (TEE) image acquired from thesubject P, on the display 32, for example. In addition, the outputtingfunction 34 e may display, as a related image, any of various medicalimages, such as an X-ray computed tomography (CT) image, a magneticresonance imaging (MRI) image, a single photon emission computedtomography (SPECT) image, a positron emission computed tomography (PET)image, and an optical coherence tomography (OCT) image, on the display32.

Furthermore, a related image related to the X-ray image I11 may be atwo-dimensional medical image generated from a three-dimensional medicalimage. For example, the outputting function 34 e may generate atwo-dimensional medical image by rendering processing of athree-dimensional medical image, such as a preoperative CT image orcone-beam computed tomography (CBCT) image, and display thetwo-dimensional medical image as a related image on the display 32. Theoutputting function 34 e may generate a two-dimensional medical image onthe basis of arm position information added to the X-ray image I11. Thatis, the outputting function 34 e my generate, as a related image relatedto the X-ray image I11, a two-dimensional medical image having animaging position and an imaging angle made to approximate those of theX-ray image I11 and display the two-dimensional medical image on thedisplay 32. The outputting function 34 e is thereby able to narrow downrelated images to be displayed to those near the real-time arm angle, ina case where display is performed during a treatment procedure, forexample.

There are plural types of related images as described above, and theoutputting function 34 e may thus narrow down related images to bedisplayed on the basis of an instruction from a user. For example, asillustrated in FIG. 5, the outputting function 34 e may cause an iconB11 and an icon B12 to be displayed, and narrow down related images tobe displayed, according to an operation on these icons. For instance,the icon B11 corresponds to “Ultrasound image”. When a user performs anoperation on the icon B11, the outputting function 34 e may selectwhether or not to display an ultrasound image as a related image relatedto the X-ray image I11.

For example, as illustrated in FIG. 6, the outputting function 34 e mayretrieve a related image using a database. For example, the eventacquiring function 34 c may perform a search in the database on thebasis of a keyword input from an input terminal, supplementaryinformation added to a acquired image, such as the X-ray image I11, orimage features of a acquired image, and display the retrieved relatedimage related to the acquired image, on a monitor, such as the display32. FIG. 6 is a diagram illustrating an example of a related imageacquiring process according to the first embodiment.

Furthermore, in FIG. 5, the detailed events (the event E121, the eventE122, and the event E123) included in the event E12 have been describedto be displayed also, but as illustrated in FIG. 7, the outputtingfunction 34 e may omit the display of these detailed events. That is,the outputting function 34 e may limit the hierarchical layers of eventsto be displayed. The outputting function 34 e is thereby able to controlthe amount of information to be provided to a user. FIG. 7 is a diagramillustrating an example of display according to the first embodiment.

Hierarchical display of events will be described below. Firstly, themanaging function 34 d manages events acquired by the event acquiringfunction 34 c by hierarchizing the events, as illustrated in FIG. 8, forexample. In the case illustrated in FIG. 8, four large events, “Start ofprocedure”, “Contrast imaging of affected part”, “Stent procedure”, and“End of procedure”, are included. FIG. 8 is a diagram for explanation ofhierarchical display of events according to the first embodiment.

An image I21, an image I22, an image I23, and an image I24 have beenassociated with the event, “Start of procedure”. Furthermore, “Start ofprocedure” includes, as a more detailed event, “Insertion of guidewire”. The image I22 and the image I24 of the images associated with theevent, “Start of procedure”, have been associated with the event,“Insertion of guide wire”. Furthermore, “Insertion of guide wire”includes, as a more detailed event, “Bifurcation check”. The image I22of the images associated with the event, “Insertion of guide wire”, hasbeen associated with the event, “Bifurcation check”. As described above,more images are associated with an event in an upper hierarchical layer(a larger event) and the number of images associated is narrowed downfor an event in a lower hierarchical layer (a more detailed event).Similarly, “Contrast imaging of affected part” includes, as a moredetailed event, “Contrast imaging of affected part”. Furthermore, “Stentprocedure” includes “Stent placement” as a more detailed event.

In addition, “Stent placement” includes, as more detailed events, “Stentpositioning”, “Dilation”, and “Check after dilation”.

The outputting function 34 e controls display on the basis of the eventshierarchically managed as illustrated in FIG. 8. For example, asillustrated in FIG. 9A and FIG. 9B, the outputting function 34 edisplays events of hierarchical layers according to an input operationreceived from a user. FIG. 9A and FIG. 9B are diagrams illustratingexamples of display according to the first embodiment.

Specifically, the outputting function 34 e firstly displays four largeevents, “Start of procedure”, “Contrast imaging of affected part”,“Stent procedure”, and “End of procedure”, as illustrated in FIG. 9A. Ina case where a user performs an operation for selecting “Stentprocedure”, for example, the outputting function 34 e additionallydisplays “Stent placement” that is a detailed event included in the“Stent procedure”. The outputting function 34 e is thereby able toprovide a detailed event according to a user's request while avoidingproviding excess information to the user.

In another example, the outputting function 34 e causes events ofpredetermined hierarchical layers to be displayed, as illustrated inFIG. 10A and FIG. 10B. FIG. 10A and FIG. 10B are diagrams illustratingexamples of display according to the first embodiment.

For example, in a case where “four layers” have been set aspredetermined hierarchical layers, the outputting function 34 edisplays, as illustrated in FIG. 10A, each of: events, such as “Start ofprocedure”, “Contrast imaging of affected part”, “Stent procedure”, and“End of procedure”, that are included in a first layer; events, such as“Insertion of guide wire”, “Contrast imaging of affected part”, “Stentplacement”, and “Check after dilation”, that are included in a secondlayer; events, such as “Bifurcation check”, “Stent positioning”, and“Dilation”, that are included in a third layer; and an event, such as“Contrast imaging check” that is included in a fourth layer. In a casewhere “two layers” have been set as predetermined hierarchical layers,the outputting function 34 e displays, as illustrated in FIG. 10B, eachof: events, such as “Start of procedure”, “Contrast imaging of affectedpart”, “Stent procedure”, and “End of procedure”, that are included in afirst layer; and events, such as “Insertion of guide wire”, “Contrastimaging of affected part”, “Stent placement”, and “Check afterdilation”, that are included in a second layer. That is, if “two layers”have been set as the predetermined hierarchical layers, the outputtingfunction 34 e may omit displaying hierarchical layers from a thirdlayer. The outputting function 34 e is thereby able to prevent confusiondue to events being too detailed. The number of the predeterminedhierarchical layers may be a preset value, a value automatically set bythe outputting function 34 e according to the total number of events, orany value set by a user.

As described above, according to the first embodiment, the imageacquiring function 34 b sequentially acquires medical images during atreatment procedure for the subject P. Furthermore, on the basis of themedical images, the event acquiring function 34 c sequentially acquiresevents in the treatment procedure, during the treatment procedure. Inaddition, the managing function 34 d manages the medical images andevents, in association with temporal information on the treatmentprocedure. Furthermore, the outputting function 34 e outputs a medicalimage such that a relation between the medical image and an event isable to be known. Therefore, the medical information processingapparatus 30 according to the first embodiment enables more effectiveutilization of medical images. For example, members of a team are ableto know the current situation and communicate with one another readily,during a treatment procedure. Furthermore, for example, if use of amedical image that has been acquired in a treatment procedure isattempted during execution of the treatment procedure, the image neededis able to be retrieved readily.

In addition, as described above, according to the first embodiment, theoutputting function 34 e may also display a related image. An imageneeded is thereby able to be retrieved readily when use of the medicalimage is attempted during execution of a treatment procedure, forexample.

Furthermore, as described above, according to the first embodiment, themanaging function 34 d manages events by hierarchizing the events. Inaddition, the outputting function 34 e displays events of a hierarchicallayer according to an input operation received from a user, or displaysevents of a predetermined hierarchical layer. The medical informationprocessing apparatus 30 is thereby able to prevent a user from gettingconfused by having too much information being displayed.

Medical images have been described as being displayed on the display 32such that relations between the medical images and events are able to beknown but the embodiment is not limited to this example. For example,the outputting function 34 e may generate the display screen illustratedin FIG. 5 or FIG. 7 and transmit the display screen to the X-raydiagnostic apparatus 10. In this case, the display illustrated in FIG. 5or FIG. 7 may be performed on the display 108 that the X-ray diagnosticapparatus 10 has.

Second Embodiment

With respect to the first embodiment, the display screens in FIG. 5 andFIG. 7 have been described as display examples. In contrast, for asecond embodiment, a display screen in FIG. 11 will be described as adisplay example. FIG. 11 is a diagram illustrating an example of displayaccording to the second embodiment. A medical information processingsystem 1 according to the second embodiment has the same configurationas the medical information processing system 1 illustrated in FIG. 1 andFIG. 2 and a part of processing by an outputting function 34 e thereofis different therefrom. Reference signs that are same as those in FIG. 1and FIG. 2 will be assigned to components that are the same as thosedescribed already with respect to the first embodiment and descriptionthereof will be omitted.

For example, as illustrated in FIG. 11, the outputting function 34 edisplays a display screen including an area R21 and an area R22 on adisplay 32. Specifically, the outputting function 34 e displays atimeline having events arranged chronologically in the area R21 anddisplays medical images acquired during a treatment procedure on thetimeline. That is, the outputting function 34 e displays a timelinehaving events arranged chronologically and displays medical images suchthat chronological relations between the medical images and the eventsdisplayed in the timeline are able to be known.

Specifically, in the case illustrated in FIG. 11, the treatmentprocedure includes an event E21, “Arrival at target region” and an eventE22, “L-ICM balloon contrast imaging”. Furthermore, in the event E21, anX-ray image I21, an X-ray image I22, and an X-ray image I23 aresequentially acquired, and in the event E22, an X-ray image I24 and anX-ray image I25 are sequentially acquired. Similarly to the cases inFIG. 5 and FIG. 7, the outputting function 34 e may also display arelated image, such as an ultrasound image, in addition to the X-rayimages acquired during the treatment procedure. Furthermore, theoutputting function 34 e may display an icon B21 and an icon B22, forexample, and narrow down related images to be displayed, according to anoperation on these icons.

Furthermore, as illustrated in FIG. 11, the outputting function 34 e maydisplay arm position information on the acquisition of the X-ray images.Specifically, the outputting function 34 e may display that the X-rayimage I21 was acquired at “LAO 45° and CAU 35°”, the X-ray image I22 wasacquired at “LAO 60° and CAU 35°”, the X-ray image I23 was acquired at“LAO 60° and CAU 30°”, the X-ray image I24 was acquired at “LAO 45° andCAU 35°”, and the X-ray image I25 was acquired at “LAO 60° and CAU 35°”.

Image reproduction is thereby able to be performed efficiently, forexample. In a case where a user refers to the display screen in FIG. 11during a treatment procedure and determines that an image having animaging angle similar to that of the X-ray image I21 is preferably usedin the current situation, for example, the user rotates a C-arm 105 to“LAO 45° and CAU 35°”. A user is thereby able to proceed with thetreatment procedure more efficiently using a real-time image of adesired angle.

The outputting function 34 e may display arm position information thathas been used for the longest period of time during a treatmentprocedure by highlighting the arm position information in the area R21of FIG. 11. That is, the outputting function 34 e may estimate anddisplay the main working angle. Such a working angle is often an angleat which useful images are able to be acquired during that treatmentprocedure and image reproduction at that working angle is neededcomparatively often. Therefore, displaying the arm position informationused for the longest time period during the treatment procedure byhighlighting the arm position information enables more efficient imagereproduction.

Third Embodiment

With respect to the first and second embodiments, the display screens inFIG. 5, FIG. 7, and FIG. 11 have been described above as displayexamples. In contrast, for a third embodiment, a display screen in FIG.12 will be described as a display example. FIG. 12 is a diagramillustrating an example of display according to the third embodiment. Amedical information processing system 1 according to the thirdembodiment has the same configuration as the medical informationprocessing system 1 illustrated in FIG. 1 and FIG. 2 and a part ofprocessing by an outputting function 34 e thereof is differenttherefrom. Reference signs that are the same as those in FIG. 1 and FIG.2 will be assigned to components that are the same as those describedalready with respect to the first and second embodiments and descriptionthereof will be omitted.

For example, as illustrated in FIG. 12, the outputting function 34 edisplays a display screen including an area R31, an area R32, and anarea R33 on a display 32. Specifically, the outputting function 34 edisplays a timeline having events arranged chronologically in the areaR31. Furthermore, the outputting function 34 e displays medical imagesacquired during a treatment procedure, in the area R32. In a case ofdisplaying X-ray images as the medical images, the outputting function34 e may display arm position information on acquisition of the X-rayimages, as illustrated in FIG. 12.

Furthermore, the outputting function 34 e displays information on inputand output performed between the outputting function 34 e and a user, inthe area R33. For example, a user inputs, through voice input or akeyboard operation, “Call a contrast imaging image.” In response to thisinput, the outputting function 34 e retrieves an X-ray image resultingfrom contrast imaging of a blood vessel of the subject P from the X-rayimages acquired during the treatment procedure, and displays theretrieved X-ray image in the area R33. As described above, theoutputting function 34 e displays a medical image according to an inputoperation received from a user and displays information in the area R33,the information being on input and output performed between theoutputting function 34 e and the user until the medical image wasdisplayed.

That is, the outputting function 34 e is artificial intelligence (AI)that receives an input operation from a user and displays a medicalimage and the outputting function 34 e may display exchanges performedbetween the outputting function 34 e and the user, in the area R33. Theuser is thereby able to readily check any image needed. For example, acommand input to call a medical image is displayed with the medicalimage in the area R33 and with what intent that medical image has beencalled is thus able to be known at a glance.

Fourth Embodiment

With respect to the first to third embodiments, the display screens inFIG. 5, FIG. 7, FIG. 11, and FIG. 12 have been described above asdisplay examples. In contrast, for a fourth embodiment, a display screenin FIG. 13 will be described as a display example. FIG. 13 is a diagramillustrating an example of display according to the fourth embodiment. Amedical information processing system 1 according to the fourthembodiment has the same configuration as the medical informationprocessing system 1 illustrated in FIG. 1 and FIG. 2 and a part ofprocessing by an outputting function 34 e thereof is differenttherefrom. Reference signs that are the same as those in FIG. 1 and FIG.2 will be assigned to components that are the same as those describedalready with respect to the first and second embodiments and descriptionthereof will be omitted.

For example, as illustrated in FIG. 13, the outputting function 34 edisplays a timeline having events, such as “Contrast imaging”,“Balloon”, and “Stent”, arranged chronologically, and also displaysplural medical images I41 such that chronological relations between theplural medical images I41 and these events are able to be known.Furthermore, the outputting function 34 e also displays supplementaryinformation for the medical images. For example, the outputting function34 e displays supplementary information, such as “Angle”, “AutoPos”, and“Vital”, for each of the medical images. These pieces of supplementaryinformation may be stored in a DICOM tag of each medical image, forexample. For example, by referring to these pieces of supplementaryinformation, a user is able to more readily find any image that the userdesires to check.

Furthermore, the outputting function 34 e may also display anexamination room interior image. For example, as illustrated in FIG. 13,the outputting function 34 e may display plural examination roominterior images I42. Specifically, during the treatment procedure, theimage acquiring function 34 b sequentially acquires examination roominterior images having captured therein the interior of the examinationroom during the treatment procedure. These examination room interiorimages are, for example, plural frames included in a moving imagecaptured using an optical camera. In addition, a managing function 34 dmanages, in addition to the medical images and the events, theexamination room interior images, in association with temporalinformation.

Furthermore, as illustrated in FIG. 13, the outputting function 34 edisplays the timeline having the events, such as “Contrast imaging,“Balloon”, and “Stent”, chronologically arranged. In addition, theoutputting function 34 e displays the plural medical images I41 suchthat the chronological relations between the plural medical images I41and the events displayed in the timeline are able to be known. Theoutputting function 34 e also displays the examination room interiorimages I42 in association with the timeline. That is, the outputtingfunction 34 e displays the events, “Contrast imaging”, “Balloon”, and“Stent”, the plural medical images I41, and the plural examination roominterior images I42 such that their horizontal axes (time series) matchone another. For example, by referring to the examination room interiorimages during execution of the procedure, the user is able to get anoverview of the procedure and the medical images are able to be utilizedfor improvement of the efficiency of the procedure.

Fifth Embodiment

The first to fourth embodiments have been described thus far, butvarious different embodiments other than the embodiments described aboveare possible.

For example, with respect to the examples illustrated in FIG. 5, FIG. 7,FIG. 11, FIG. 12, and FIG. 13, the case where at least a timeline havingevents arranged chronologically is displayed has been described.However, the embodiments are not limited to this case. For example, theoutputting function 34 e may omit the display of a timeline, and maydisplay medical images associated with an event received from a userthrough voice or a keyboard, for example.

Furthermore, for example, with respect to the embodiments describedabove, the case where the processing circuitry 34 of the medicalinformation processing apparatus 30 executes functions, such as theevent acquiring function 34 c, the managing function 34 d, and theoutputting function 34 e has been described. However, the embodimentsare not limited to this case, and for example, the processing circuitry110 of the X-ray diagnostic apparatus 10 may execute functionscorresponding to the event acquiring function 34 c, the managingfunction 34 d, and the outputting function 34 e.

For example, the processing circuitry 110 further executes an eventacquiring function 110 d and a managing function 110 e not illustratedin the drawings. For example, the acquiring function 110 b sequentiallyacquires X-ray images on the basis of X-rays transmitted through thesubject P during a treatment, procedure for the subject P. Furthermore,on the basis of the X-ray images, the event acquiring function 110 dsequentially acquires events in the treatment procedure, during thetreatment procedure. Furthermore, the managing function 110 e managesthe X-ray images and the events, in association with temporalinformation on the treatment procedure. The outputting function 110 cthen outputs the X-ray images such that relations between the X-rayimages and the events are able to be known. For example, the outputtingfunction 110 c may display the display screen in FIG. 5, FIG. 7, FIG.11, FIG. 12, or FIG. 13, on the display 108.

The acquiring function 110 b may acquire X-ray images again on the basisof a acquisition condition for an X-ray image that has been selected bya user from the X-ray images output by the outputting function 110 c.For example, in a case where the outputting function 110 c displays thedisplay screen in FIG. 11 on the display 108 and a user selects theX-ray image I21, the acquiring function 110 b acquires X-ray imagesagain for the condition, “LAO 45° and CAU 35°”. In a case where apredetermined operation button is pressed while the X-ray image I21 isbeing displayed in the area R22, for example, the acquiring function 110b determines that the X-ray image I21 has been selected and executesacquisition of X-ray images for the condition, “LAO 45° and CAU 35°”.Image reproduction is thereby able to be performed more efficiently.

The term, “processor”, used in the description above means, for example,a circuit, such as a central processing unit (CPU), a graphicsprocessing unit (GPU), an application specific integrated circuit(ASIC), or a programmable logic device (for example, a simpleprogrammable logic device (SPLD), a complex programmable logic device(CPLD), or a field programmable gate array (FPGA)). In a case where theprocessor is a CPU, for example, the processor implements the functionsby reading and executing programs stored in a storage circuit. In a casewhere the processor is, for example, an ASIC, instead of the programsbeing stored in a storage circuit, the functions are directlyincorporated, as logic circuits, in the circuit of the processor. Eachof the processors according to the embodiments is not necessarilyconfigured as a single circuit, and plural independent circuits may becombined together to be configured as a single processor to implementtheir functions. Plural components in each drawing may also beintegrated into a single processor to implement their functions.

The components of each apparatus according to the embodiments describedabove have been functionally and conceptually illustrated in thedrawings and are not necessarily configured physically as illustrated inthe drawings. That is, specific forms of distribution and integration ofthe apparatuses are not limited to those illustrated in the drawings,and all or a part of each apparatus may be configured to be distributedor integrated functionally or physically in any units, according tovarious loads and/or use situations, for example. In addition, all orany part of the processing functions executed in the apparatuses may beimplemented by a CPU and a program analyzed and executed by the CPU orimplemented as hardware by wired logic.

Furthermore, any medical information processing method described abovewith respect to the embodiments may be implemented by a computer, suchas a personal computer or a work station, executing a medicalinformation processing program that has been prepared beforehand. Thismedical information processing program may be distributed via a network,such as the Internet. Furthermore, this medical information processingprogram may be recorded in a computer-readable non-transitory recordingmedium, such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, or aDVD, and executed by being read by a computer from the recording medium.

According to at least one of the embodiments described above, medicalimages are able to be utilized more effectively.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

1. A medical information processing apparatus, comprising: processingcircuitry configured to: sequentially acquire medical images during atreatment procedure for a subject; sequentially identify images in whichevents have occurred, from the medical images sequentially acquired,based on the medical images sequentially acquired; and store the medicalimages acquired and also stores information on the events that haveoccurred, in association with the identified images in which the eventshave occurred, the identified images being from the medical imagesstored.
 2. The medical information processing apparatus according toclaim 1, wherein the processing circuitry is further configured to causea display to display the stored medical images, such that chronologicalrelations between the stored medical images and the events that haveoccurred are able to be recognized.
 3. The medical informationprocessing apparatus according to claim 2, wherein the processingcircuitry is further configured to cause the display to display atimeline having the events arranged chronologically based on thetemporal information and also display the medical images such thatchronological relations between the medical images and the eventsdisplayed in the timeline are able to be known.
 4. The medicalinformation processing apparatus according to claim 3, wherein theprocessing circuitry is further configured to cause a related imagerelated to the medical images to be displayed.
 5. The medicalinformation processing apparatus according to claim 4, wherein theprocessing circuitry is further configured to cause the related imagerelated to the medical images to be displayed, wherein the related imageis an image having supplementary information that is the same as orsimilar to that of the medical images or an image having image featuressimilar to those of the medical images.
 6. The medical informationprocessing apparatus according to claim 4, wherein the processingcircuitry is further configured to cause the related image related tothe medical images to be displayed, wherein the related image is animage acquired using an apparatus different from an apparatus for themedical images.
 7. The medical information processing apparatusaccording to claim 4, wherein the processing circuitry is furtherconfigured to cause the related image related to the medical images tobe displayed, wherein the related image is a processed image that hasbeen generated based on the medical images.
 8. The medical informationprocessing apparatus according to claim 4, wherein the medical imagesare X-ray images acquired using an X-ray diagnostic apparatus, and theprocessing circuitry is further configured to cause the X-ray images tobe displayed such that relations between the X-ray images and the eventsare able to be known and also cause an X-ray image to be displayed, theX-ray image having arm position information that is the same as orsimilar to that of the X-ray images.
 9. The medical informationprocessing apparatus according to claim 4, wherein the medical imagesare X-ray images acquired using an X-ray diagnostic apparatus, and theprocessing circuitry is further configured to cause the X-ray images tobe displayed such that relations between the X-ray images and the eventsare able to be known, and also cause the related image to be displayed,the related image being a two-dimensional medical image that has beengenerated from a three-dimensional medical image based on arm positioninformation of the X-ray images.
 10. The medical information processingapparatus according to claim 4, wherein the processing circuitry isfurther configured cause the related image to be displayed by processingor generating the related image according to the medical images.
 11. Themedical information processing apparatus according to claim 3, whereinthe medical images are X-ray images acquired using an X-ray diagnosticapparatus, and the processing circuitry is further configured to causethe X-ray images to be displayed such that relations between the X-rayimages and the events are able to be known and also cause arm positioninformation to be displayed, the arm position information beinginformation on an arm position at the time of acquisition of the X-rayimages.
 12. The medical information processing apparatus according toclaim 3, wherein the processing circuitry is further configured to causethe medical images to be displayed according to an input operationreceived from a user and also cause information to be displayed, theinformation being on input and output performed between the processingcircuitry and the user until the medical images were displayed.
 13. Themedical information processing apparatus according to claim 3, whereinthe processing circuitry is further configured to: sequentially acquire,during the treatment procedure, examination room interior images having,captured therein, the interior of an examination room during thetreatment procedure; manage the medical images, the events, and theexamination room interior images, in association with the temporalinformation; and cause a timeline to be displayed based on the temporalinformation, the timeline having the events arranged chronologically,cause the medical images to be displayed such that chronologicalrelations between the medical images and the events displayed in thetimeline are able to be known, and cause the examination room interiorimages to be displayed in association with the timeline.
 14. The medicalinformation processing apparatus according to claim 13, wherein theprocessing circuitry is further configured to cause the medical imagesto be displayed with supplementary information of the medical images.15. The medical information processing apparatus according to claim 1,wherein the processing circuitry is further configured to acquire anevent in the treatment procedure based on a database storing eventtypes.
 16. The medical information processing apparatus according toclaim 3, wherein the processing circuitry is further configured to:manage the events by hierarchizing the events; and cause the events tobe displayed, the events being of a hierarchical layer corresponding toan input operation received from a user.
 17. The medical informationprocessing apparatus according to claim 3, wherein the processingcircuitry is further configured to manage the events by hierarchizingthe events, and the display displays the events of a predeterminedhierarchical layer.
 18. An X-ray diagnostic apparatus, comprising:processing circuitry configured to: sequentially acquire X-ray imagesbased on X-rays transmitted through a subject during a treatmentprocedure for the subject; sequentially acquire events in the treatmentprocedure on the basis of the X-ray images during the treatmentprocedure; manage the X-ray images and the events, in association withtemporal information on the treatment procedure; and output the X-rayimages such that relations between the X-ray images and the events areable to be known.
 19. The X-ray diagnostic apparatus according to claim18, wherein the processing circuitry is further configured to executeacquisition of X-ray images again based on an acquisition condition foran X-ray image selected by a user from the output X-ray images.
 20. Anon-transitory computer-readable medium storing a medical informationprocessing program that, when executed, causes a computer to execute amethod comprising: sequentially acquiring medical images during atreatment procedure for a subject; sequentially acquiring events in thetreatment procedure based on the medical images during the treatmentprocedure; managing the medical images and the events, in associationwith temporal information on the treatment procedure; and outputting themedical images such that relations between the medical images and theevents are able to be known.